A concept virtual harp with physical string vibrations using augmented reality for therapy

This thesis presents a multimodal three-dimensional harp for interactive musical experiences for use in therapy for individuals with disabilities. A preliminary harp model in OpenGL with user interaction being a mouse and keyboard tested on special education children lead the Virtual Harp program on a new design approach to provide more interaction and stimuli. This system provides individuals with three forms of feedback visual, audio, and haptic during interaction using a Sensable Phantom Omni haptic device and immersion in a portable augmented reality-based system using a video see-through head mounted display. Modeled as a realistic harp, the virtual harp uses realistic harp string vibrations modeled from Fourier analysis physics equations. The individuals play the virtual harp with the Phantom Omni haptic device by plucking or strumming the strings of the harp. They have the freedom to move the harp around the environment giving the feeling of playing a traditional musical instrument. A virtual therapist feature was developed to allow a therapist without musical knowledge to observe the individual during therapy exercises and allow the individual to engage in a self-motivating therapy exercises outside the therapy room by following a simple sequence of notes by following the color-coded strings. User testing was performed to measure usability and therapeutic effectiveness: development of skills, improvement in range of motion, and entertainment value. Results showed that there were no major usability issues discovered and responses were positive with regards to using this technology for therapy.